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Multicast Control Systems and Methods f r 
Dynamic, Adaptiv Tim and Bandwidth Allocations 
Related Applications 

This invention claims priority to U.S. provisional application serial no. 
60/129,988, filed April 18, 1999, which application is hereby Incorporated in its 
entirety by this reference. 

Field of the Invention 

The present invention generally relates to satellite data networi<s. and more 
particulariy, to systems and methods for managing two-way satellite data networi^ 
systems through a centralized central server utilizing a terrestrial link. Methods and 
systems are also disclosed for dynamic bandwidth allocation using a hybrid rate 
allocation and time-slicing mechanism with distributed, hierarchical control. 

Background of the Invention 

Data communication via satellite has grown with leaps and bounds recently. 
That growth has reduced transmission costs and increased usage - but at the cost 
of bandwidth congestion. In part the congestion is due to an operation environment 
that utilizes static channel assignments with scheduling based on manual time 
reservation mechanisms. 

For example, In the past, distiibuting television, cable and otiier video/sound 
files to local stations around the country has been done by manually scheduling tiie 
feeds. Typically, a satellite transmitter has a fixed allocation of bandwidtii and, if a 
user needs to ti^nsmit 10 minutes worth of data, the user would manually place a 
phone call to a provider and say I would like 10 minutes of ti-ansponder time at 4:00 
p.m. This negotiation for ti-ansponder time is time consuming and inefficient and 
may result In the user being forced to purchase an hour's worth of transponder time 
when all he needs is 10 minutes. CJonsequentiy, once the user finishes transmitting 
his data, tiie transponder sits domiant for tiie remaining 50 minutes, which results in 
wasted capacity. Currentiy. no providers have a system ttiat promotes the efficient 
use of transponder time tiirough tiie re-use and sharing of previously dedicated 
facilities. 

Some alternative satellite providers are looking at reliable multicast 
transmission to be a key technology tiiat can be used to minimize network traffic. 
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Coordination is a major problem with multicast file distribution over a satellite. In 
other words, how can many content providers (e.g., channels, news organizations, 
etc.) distribute their content witiiout ovenwhelming ttie network bandwidtii? This is 
especially a problem where bandwidth allocation is handled ttirough a manual 
system. 

A second problem in multicasting files over a satellite communications 
networi< Is that you cannot determine exactiy when your file is sent. No one can 
know witti certainty when a particular transmission will end. In the past, people have 
dealt witii this problem by^statistically evaluating ttie data transmission infonnation to 
determine a statistical end point over time. That is, by looking at a number of 
transmissions after the fact, one can figure out how many actually went tiirough on 
time. Usually, that percentage is very high - in ttie 90% range. It can be made even 
higher, but the ti^ade-off typically is a less efficient use of satellite h^ansponder time. 
In other words, if you want to really know that your file is going to be sent over tiie 
satellite when you schedule it. you need, overall, to schedule less traffic than the 
system can handle. In essence, this is very like what a doctor does in scheduling 
patient appointments - if the doctor wants to make sure he sees ttie maximum 
number of patients she will schedule lots of appointinents over tiie day. But, as the 
day progresses, the delay between when the doctor sees ttie patients and tiie 
patients' appointment times lengfliens. The doctor can fix tiiis by reducing tiie 
number of appointironts, but tiien ttie doctor mns ttie risk of having some down time 
during which no patient is scheduled. 

In any event if bandwidtii is allocated based on these statistical averages, 
inevitably some jobs will not be completed wrOiin the allocated transmission time. 
Skilled persons in tiiis area say tiiis means fliat your file completion times are "non- 
deterministic." The nonnJetermlnistic file transmission causes scheduling diffioilties 
in projecting future start times for ottier date transmission. This can be a critical 
problem. For instance, consider a breaking news story, such as a presidential news 
(inference, tiiat a networic wants to make sure gete out across ttie counfry to 
various local affiliates. It is critical that ttie networic knows ttiat at flie set time tiie 
news story will be tinansmitted. You would normally tiiink ttiat to guarantee files are 
sent on time, and thus solve the scheduling problems described above, one could 
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just increase the rate the data is sent over the satellite. But this brings up another 
problem \Artth multicast - data throughput varies. For example, suppose live out of 
six cable headends where the cable equipment operators maintain their equipment 
have state of the art, well maintained equipment that can receive data at 100Mbps 
(one MBps is equivalent to one million bits of data per second). One of the six head 
ends can only receive data at 10Mbps. In this situation, if all six need to receive the 
file within a certain designated time, increasing transmission bandwidth will not 
guarantee a transmission stop time because there is a "weak link in the chain." 
namely the 10 Mbps head end. Thus, transmissions must be made using a 
"dynamic" allocation of bandwidth and transmission time that accounts for the 
varying throughput of various system components. An allocation is dynamic if the 
system can change the time slice or data rate of the transmission. 

This invention provides a distributed-hierarchical scheduling control system 
that monitors the networic and adapts dynamic rate allocation, time slicing and 
logical framing to optimize system resources. 

Summary of the Invention 

This invention addresses these issues by introducing dynamic rate allocation, 
dynamic time slicing and logical framing, and a distributed, hierarchical control 
system. According to tfie methods and systems of tiiis invention, adaptive network 
programming optimizes resource allocation. This document briefly describes 
systems and methods for tiie effective sharing of a satellite transponder between 
multiple satellite uplink stations using out-of-band ten^estrial signaling/continol. Using 
an out of band communication link such as a separate communications network like 
the Internet, VSAT. or frame relay witii an efficient protocol, b^nsponder sharing 
among satellite uplink stations can be controlled and automated. The automation 
process will minimize idle time, which means additional satellite capacity for the 
users. 

The present invention advantageously enables a disti'lbuted set of uplink 
station facilities to effidently share an outtDound satellite link witiiout requiring tiie 
expense of two-way RF or satellite transmission. Requests and resulting channel 
assignments are specified in terms of scheduled transmission times witiiin a variable 
time horizon. Accordingly, tiie system can offer either a scheduled batch 
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transmission semce mode or an Interleaved, time division multiple access mode. 
The master arbitration sen/er provides a master clocking reference for 
synchronization. This clocking reference may be generated from a highly accurate 
local reference (e.g., an atomic clock) or may be slaved to an external source, such 
as the GPS network. 

This invention effectively shares a limited amount of satellite resources with 
multiple remote sites and satellite transmitters. Using an out of band control in 
accordance with the present invention supports use of the satellite transponder for 
either analog video or high-speed data transfers. Two way data communication is 
also possible using the satellite for ttie outgoing high-speed data transfer and the 
control patii as ttie return data path, and "path-switching" between two or more 
transmission sites in a half-duplex communications fashion. 

Among other objectives, tiiis invention aims to overcome the problems of the 
current environment of satellite transmissions that use static channel assignments, 
with scheduling based on manual time reservation mechanisms. The non- 
detemninistic file distribution predictions available under prior systems causes 
scheduling difficulty In ttiat future start times cannot be precisely predicted. On the 
other hand, because multicast throughput is variable, multicast file distiibution that is 
based only on changing the rate or band width of the connection still does not solve 
the problem. 

In one embodiment, this Invention uses a system of master and sub- 
controllers to overiay, adaptively, dynamically or both, rate and time slice allocations 
upon a common set of nefwori^ transport resources. These network transfer 
resources may be in tiie fomn of wireline, wireless, satellite or otiier communications 
networi< capacity. A master arijitration server receives access requests from 
multiple local (or remote) controllers, each of which desires to schedule a file 
transmission over a communications resource such as a satellite. The access 
requests Include infomiation that tiie master arbitration server uses to assess ttie 
priority of the file transmission. A schedule for satellite use is set based on this 
information. The master arbitration server returns to each local ari^itration server 
information concerning tiie time slice to which they will have access to tfie satellite 
transponder for using satellite capacity. 
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Local arbitration servers, in turn, have authority over their use of the time slice 
assigned for their use. The local arbitration servers may subdivide their time slices 
based on their local needs and the priorities of the files they need transfen-ed, 
resulting in a file transmission scheme that \s similar to a time division multiplexed 
scheme. In addition, however, the local arbitration servers also may set the file 
transmission rate (e.g., bandwidth) to be assigned a particular transmission. In 
essence, this system provides an overlay of time slicing and rate allocation by which 
the distributed control system can specify a window of time and a rate in order to 
ensure better file transmission. 

Another advantage to using local arbitration servers is the distributed 
intelligence achieved in the system allows for dynamic reaction to events. This Is 
best explained by example. Past transmission system schemes have static 
retransmission schema. Thus, the system might try to send a file 7 times before 
giving up. The problem with this approach is that the static schemes add to networi< 
rigidity. The system can become unstable because it receives problem jobs that 
require retries and then either delay other jobs, causing a ripple effect on the 
schedule, or simply do not get sent Meanwhile, new jobs are entering the system 
for transmission. The local arbitration servers of this invention allow the system 
dynamically to decide whether to keep trying to send a particular file or job or 
whether to move on to the next file. Also, the local art)itration servers can decide to 
send part of the file now and hold the rest for transmission in another time slice, 
resulting in tiie local art)ita^tion server effidentiy using some of ttie assigned capacity 
and getting at least a portion of the file complete. By using a system having a 
master arbitration server and a number of local art^itration servers, tills Invention 
promotes efficient reuse and sharing of previously dedicated networic transport 
resources. 

The invention may be adapted for using an internet protocol f IP") multicast- 
based service for distributing various files. This service achieves large-scale 
distribution of the files, such as media files comprising shows, commercials, news 
events, sport events, etc., via an IP multicast witiiin an adaptive reliability layer. The 
master and local arbitration servers bounce tiie use of network transport resources 
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with the respective priorities and service commitments associated with each file or 
set of files that need to be distributed over the next system. 

This system may be implemented by coupling a master ariDitration server to a 
communications network that allows the master arbitration server to communicate 
with various local arbitration servers, each of which are associated with a portion of 
the network transport facilities. For instance, a local arbitration server may be 
associated with a satellite transmitter and the appropriate hardware necessary to 
package the data for satellite transmission. The master arbitration server may 
comprise a scheduling server that sets the basic distribution times for a file or set of 
files. The master ariDitration server may assign a time slice and channel to a 
particular file. The master arbitration server forwards that assignment data to the 
local arbitration server. The local arbitration server, depending upon the other jobs it 
receives and/or networi< traffic data, will itself adaptively assign each of the multiple 
jobs a particular rate and, if appropriate, may further divide the time slices to which 
each job is assigned. The local arbitration server thereby dynamically and 
adaptively overiays rate allocation upon the initial time slice assigned by tiie master 
arbitration server. 

AnoUier aspect of Oiis invention involves detemnining flie rate at which files 
may be broadcast As skilled persons will recognize, botUenecks in multicasting 
occur on communication links. Additionally, however, bottienecks occur because of 
the servers and clients receiving the files. For instance, each time a client computer 
opens a new application, tiiat application detects firom ttie client computer 
resources othenA^ise available for accepting files, which, in turn, decreases the rate 
tiiat the client computer can receive file ta^nsfers. By providing feedback on the 
data rates handled by portions of the communications network, including botii the 
communications link and the clients receiving tiie files, the present invention can 
furttier improve multicast file transfer performance. 

Accordingly, using tiie methods and systems of this Invention achieves at 
least one, more or combinations of the following advantages. The invention: 
• Can maximize the number of files ti'ansmitted to the maximum number of 

sites successfully at the highest rate possible. 
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• Utilizes two degrees of freedom (rate and time) to manage priority and non- 
priority data transfers. 

• Provides best-effort and guaranteed service levels. 

• Utilizes distributed control and hierarchical bandwidth management agents. 

• Utilizes networt< management statistics to adapt bandwidth allocation. 

• Can emulate rate-proportional sharing (TCP/IP) for IP multicast 

• Can be deployed above standard RF media access control mechanisms. 
Other objects, features and advantages of this invention will become apparent 

from the rest of this document, including the Figures. 

Brief Description of the Drawings 

Fig. 1 is a diagram showing an exemplary topology for one embodiment of the 
system of the present invention. 

Fig. 2 provides additional detail of system components and resources that 
can be used to implement the embodiment shown in Fig. 1. 

Fig. 3 is a schematic diagram showing the intenrelationships between the 
local arisitration servers and the master arisitration sen/er of the present invention. 

Fig. 4 is a logic diagram depicting one metiiod for managing the channel, 
rate-time and band width hierarchy of various files to be multicast according to the 
present invention. 

Fig. 5 is a broad overview showing the logical coupling point of several 
components of the system of the present invention, 

Rg. 6 is an overiay depicting the dynamic rate-time allocation for multicast file 
distribution that is accomplished according to tiie metiiods and systems of this 
invention. 

Fig. 7 depicts the priority queuing and bandwidth allocation perfomied by the 
multicast server according to the mettiods and systems of tiie present invention. 

Fig. 8 is a high level diagram depicting the interrelationship between the 
master arbitration server and various local arbitration servers. 

Fig. 9 depicts one request-grant protocol for implementing aspects of the 
present Invention. 
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Fig. 10 is a logic diagram depicting the adaptation medianism by wiiich the 
methods and systems of the present invention adapt to varying networl< conditions in 
order more efficiently to distribute files. 

Detail d Description of the Drawings 

Before describing ttie drawings and embodiments in more detail, several 
temis are described below in an effort to ciarily the terminology used in this 
document. Additional and fuller understanding of these temris will be clear to 
persons skilled in this art upon reading this entire document 

• lOObaseT: Ethernet network 100 megabit per second interconnection. 

• Client: A client is a program, which interacts with a Server program that 
may be on a remote machine. 

• CRC: Cyclic Redundancy Check - a type of checksum used to validate 
data, generally after storage or transport. 

• DNS: Domain Name System 

• DVB: Digital Video Broadcast 

• EDI: Electronic Data Interchange 

• FDM: Frequency Division Multiplexing 

• IP: Internet Protocol 

• Local Arbitration Server this is a computer, such as a woricstatlon, which 
runs software to allow it to detemnine which jobs to put into the queue on 
what schedule and on what bandwidth. The local arbitration server has a 
view of the demand for the resources locally. 

• Master Arbitration Server, this is a computer, such as a workstation, 
running network control software that manages overall resources and has 
a scheduling function. The master arbitration server allocates channels, 
an aggregate rate and/or an aggregate time slice. The master arbitration 
server "arbitrates" between the requests for satellite resources made by 
various content providers, the network capabilities and the local arbitration 
servers. 

• Multicast: a process of sending a transmission to a selected subset of 
recipients (as opposed to point to point, or broadcast). 
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• MPEG2: Second generation digital video compression and transmission 
standard fomialized by the IWoving Pictures Experts Group. 

• NT: Short for Windows NT Server, an operating system from Microsoft 
Corporation. 

• Out of Band: Not transmitted with the signal in question. May be on a 
different frequency, or by another means altogether (e.g. Internet vs. 
satellite). 

• PIDs: Program Identifiers as defined by MPEG. 

• Priority Infonnation: this refers to information that the arbitration sen/ers 
may use In order to assign a particular priority to a job. The Information 
can be as simple as a pre-assigned priority code and delivery time to more 
detailed information, such as the credit worthiness of Uie sender, the 
volume of business provided by the sender, the value of the infonnation, 
whether the sender will pay a surcharge for a particular service guarantee, 
etc. With such information in hand skilled persons will recognize ttiat the 
servers or associated programs may assign a priority. An additional foctor 
to note conceming priority infonnation is tfiat while a request may contain 
some priority information, oflier priority information may be stored at the 
server and used in assigning the ultimate priority. 

• QPSK: Quadrature Phase Shift Keying 

• Scheduler. A program that guarantees tiiat requested content has been 
successfully transmitted as scheduled. 

• Server A program fliat provides specific services to clients (e.g. database 
or protocol management). It is frequentiy is important enough (or busy 
enough) to run alone on a dedicated machine. 

• Signaling Channel: A separate channel used to conb'oi activity on otiier 
channels, or otiier processes. 

• TDM: Time Division Multiplexing 

• Uplink station: tills refers to an aggregation of equipment for transmitting 
(or even receiving). Each station may contain data communications 
equipment for communicating with a networi< or computer to receive files, 
upconversion equipment, an RF amplifier and Uie actual ti^nsmit dish. 
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• WWW: World Wide Web. an Internet hypennedja application. 
Overvi w Of The Present Invention (Figs. 1-3): 

Fig. 1 depicts various components comprising a portion of the architecture of 
the system of this invention. A master control network 100 allows a master 
arbitration server 1 1 0 to control various local arbitration servers 120. Each of the 
local arbitration sen/ers 120 is coupled to a local multicast server 122 and to a 
satellite transmitter 130. Each of the local multicast servers 122 store the content 
that is to be distributed via the network transport facilities such as the satellite 
network shown in Figure 1. In concept, Figure 1 shows a distributed, hierarchical 
system in which each satellite transmitter 130 has a set of clients comprising various 
satellite receivers 150. The satellite transmitters 130 in turn have local multicast 
servers 122 that acts as a scheduler and a local ariDitration server 120. Note that 
the functionality of the multicast servers 122 and bandwidth manager 120 can be 
implemented on one or multiple wori^stations or computer platforms. 

The satellite transmitter 130 communicates via satellite 140 to various 
satellite receivers 150 that couple via remote backchannel 160 to the local arbitration 
servers 120 and master bandwidth manger 110. Master arbitration server 110 
couples to a database 1 12 supplied with networic management data via a networt< 
management subsystem 1 14. The master control network 100 allows for out-of- 
band interconnection between various components of the system shown in Fig. 1. 
For instance, the master control network 100 can be a communications network 
configured for TCP/IP, VSAT, frame relay. ATM. or internet communications. 
Similariy, remote backchannel 160 can be a backchannel for receiving downlink 
infonnation (e.g.. confirmation of file receiptAransmission status) that can be a 
communications networic such as a packet switched network or the internet. 

Figure 2 shows one implementation of the architecture shown in Figure 1. 
Figure 2 shows an implementation of the control system of the present invention in 
which multiple satellite uplink stations 210 share unidirectional satellite 202 
bandwidth by scheduling access for uplinking earth stations 210 via a central server 
220 placed in a centralized location for storing video data 222. Each of the uplinking 
earth stations 210 has access to the central server 220 through the Internet (or 
some other suitable communications link). The central server 220 may control the 
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allocation of bandwidth within one transponder, the allocation of multiple 
transponders on one satellite, and/or the allocation of bandwdth across multiple 
satellites 202. 

Each uplink station 210 comprises satellite transmission and fonnatting 
equipment known to persons skilled in the art. Associated with the uplink stations 
210 are one or more local multicast servers 212, 214, each of which provides the 
access control at the earth uplink station 210 and local storage for data 222 to be 
transmitted over the satellite 202. The multicast servers 212. 214 have access to the 
Internet (or other suitable communications pathway) for communications with the 
central server 220, via a local signaling server. In addition, the uplinking earth 
station 210 site may have satellite modem equipment connected to the local 
multicast server and satellite earth station transmission equipment The central 
server 220 will schedule access for any individual, local multicast sender 212, 214, 
placed at any uplinking earth station 210. An illustrative embodiment of a control 
system architecture in accordance with the present invention is illustrated in Figure 2 
wherein two of a possible plurality of uplinks are shown. 

Each of the local multicast servers 212, 214 may store packetized data in 
files. Each file can be selected individually for transmission or grouped together for 
transmission in a batch mode. Once a file has been selected for transmission, a 
selected protocol is followed far transmission. 

For instance, the local multicast server 212 may request a transmission 
virindow from the central sen/er 220. master arbitration server 220 prioritizes 
communications across the satellite 202 resources available. Access to the satellite 
202 resources can be granted on a "first-come first-serve basis" or any number of 
priority schemes to provide a particular guaranteed quality of sen/ice (QOS). The 
central server 220 also may incorporate a bandwidth management system, which 
makes sure that the available satellite resource is never over-subscribed. In one 
protocol according to this invention, the central server 220 may provide the 
requesting local multicast sender 212 a message detailing the transmission window 
for the particular file transmission. The central server 220 authorizes the actual 
transmission over the satellite 202 by sending an authorization message for the 
designated satellite resource to the local multicast sen/er 212 that is scheduled to 
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transmit No local multicast server may begin transmitting data through the satellite 
resource unless it has received the authorizing message, and not until the 
authorized start time. The control system uses authorizing messages as the 
methodology to insure that only one uplinking earth station is accessing the satellite 
202 resources at any given moment. 

After it has received the authorizing message from the central server 220. the 
multicast server 212 manages the transmission of the data through the satellite 
transponder. At the exact authorized time, the carrier for the uplink station's 210 is 
enabled and the output of the multicast server 212 feeds to the uplinking earth 
station's 210's modulation equipment After it is modulated, it is then upconverted 
and amplified for transmission through the satellite 202 transponder. Any number of 
applications can be used to manage the flow of data over the satellite. Protocols 
known to skilled persons in this field can be employed to selectively retransmit data 
that was corrupted before being received by the earth stetions. 

The multicast server uses the Intemet (or another suitable communications 
pathway) for communications with the receive stations or "clients." After it schedules 
a transmission with the central server 220, it contacts the client sites 230 through an 
open multicast group address. The open group is contacted and the members of 
the smaller transmission group are instmcted to begin "listening" to a satellite 
resource for the transmisston. Once switched to the new closed group address each 
client in the newly fonned "closed group" is instmcted to register v\nth the multicast 
server 212. 214 that initiated the session. The server's IP address for this response 
is passed during the registration process. This response occurs over the Intemet (or 
another suitable communications path for return data) and that client (or receive site 
232, 234) can now begin to receive the multicast data being distributed in that 
particular dosed session. 

While a local multicast sender 212, 214 is authorized to transmit, the central 
server 220 may send it an update message allocating increased or reduced 
bandwidth and / or changing Its frequency assignment It may even ask the local 
multicast server 212 to pause its transmission. The multicast server 212 then takes 
appropriate steps to cause its group to make this change, and then retum an 
acknowledgment message to the central server 220. This allows the central server 
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220 to handle higher priority QOS requests, and to prevent the fragmentation of 
bandwidth within a transponder. If the multicast server 212 finishes eariier than its 
allotted time, it may notify the central server 220 of its intent to relinquish the time 
remaining. This allows the central server 220 to reallocate that time. It must leave 
its earner on until it receives an updated authorization message fi^om the central 
server 220 telling it the exact time that it must turn off its earner. It may request 
additional time in the same manner, but the transaction granting additional time must 
be completed before the end time of the original authorization. The multicast server 
212 should shut down its canier at the exact time specified in the cun^ent 
authorization message. 

Figure 3 presents a summary of the architecture of one system for 
implementing this invention. A master arbitration server 300, which artjitrates 
amongst the various requests for satellite uplink time and the local arbitration 
servers 302, couples via a communications networi< 310 to various multicast servers 
320 located remotely from the arbitration server 300. Through an IP gateway 330, 
the servers 320 push content into a standard DVB modulator 340 for uplinking over 
equipment 350 on the schedule and at the rate dictated, as described below, by the 
master and local ari^itration servers. The uplink station has various components 
including the modulator 340, uplink equipment 350, or dish 360 for transmitting files 
to satellite 360. 

Dynamic Bandwidth and TimeSlice Allocation (Figs. 4-9): 

Figure 4 shows a logical overview of a method of this invention for 
dynamically allocating files rates and times for transmission over network transport 
resources such as a satellite. Figure 4 depicts a distributed control hierarchy In 
which the steps in area 420 are canied out by a master ari^itration server 410 and 
the steps in area 420 are canied out by the local arbitration servers. Master 
arbitration server 410 receives multiple requests for distributing content and assigns 
each request a specific channel, represented by firequencies F1, F2, Fn. Each 
channel has a number of time slots to which the master ari^itration server 410 
assigns particular Jobs. Thus, master arbitration server 410 subdivides channel 1 
into time slots T1 1, T12, Tin, in which the first subscript represents the channel and 
the second represents the time slice. The master arbitration server 410 may 
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maintain a database describing the capacity on each channel available to the 
network. 

Local arbitration servers receive from the master arbitration server the initial 
time and channel infomiation, as well as an initial amount of bandwidth. Therafter, 
each local arbitration server assigns its jobs a rate and may further subdivide the 
jobs into different time slots and at different rates to make more efficient use of the 
resources allocated to that local arbitration server. Thus. Figure 4 shows in area 
430 where local arbitration servers add rate designator 1 to channel 1, time slot 1, 
resulting in a T1 1 1 to T1 1n different rates applied to the first time slot in the first 
channel. By way of example, suppose in Figure 4 that the master arbitration server 
410 assigns a 30 second time slice to channel 1. Hierarchically, as seen in Figure 5, 
the next level is controlled by the local arbitration servers 520, each of which 
communicate with the master arbitration server 510 (the same as server 410 in 
Figure 4) and various clients 530, 532 that comprise, e.g., the uplink stations or local 
multicast servers. Each local arbitration server 520 assigns a rate and may further 
subdivide the time slot into two slots of 15 seconds apiece at a rate of 5 Mbps each. 
Without the further time slicing, the system would be at the mercy of the random 
errors always present in file transmission, with the result that an error may occur and 
delay the job. 

Figure 6 attempts to capture the degrees of freedom the hierarchical system 
off this invention provides in allocating rate and time to multicast file distribution. The 
columns shown in Figure 6 represent the different rates assigned whereas the rows 
represent tiie time slots. So, for each frequency, tfiere is rate and time multiplexing 
of tiie IP packets in which the files are subdivided. Rgures 7 and Ssimilariy shows 
how the local and master arbitration senders 720 and 722 (or 810, 820 in Rgure 8) 
interact witii ttie local multicast sender 710 (which provides the content for 
transmission) in order to create a priority queue comprising the rate and time 
allocation for various Jobs. 

Figure 9 depicts a particular implementation of a request/grant protocol by 
which the local arbitration servers inform the master of ttieir need for new 
allocations. As seen in Figure 9, in the Transmission Setup phase, the client that 
desires to send the job first sends a Request In tiiis instance, the client may be the 
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local arbitration server, which has received a request for file transmission and priority 
information associated therewith. The request from local to master arbitration server 
provides a Queue Descriptor detailing priority information on the job, as well as a 
Delivery Request indicating completion time. The master ari^itration server, which 
knows of the demands on the overall networi^, receives multiple Requests from 
multiple clients. It analyzes, prioritizes and schedules the requests based on 
network resources, performance and request priority data. It then forwards to each 
of the local ariDitration servers a Request Response setting the rate, timeslice, 
channel ID (e.g., frequency) and a transaction ID for tracking purposes. 

Next, the transmission is scheduled and made by the local arbitration server 
and associated equipment. It may be that during or after the transmission the 
master art^itration server may recognize that the local arbitration server needs more 
capacity to complete its particular job. For example, the master arbitration server 
rnay have sent the local arbitration server only enough networi^ transport capacity to 
complete part of the job in question, noted that deficiency and looked for additional 
system resources to allow transmission of the remainder of the file. Or, the master 
arbitration server may have received infomnation concerning resource performance 
in the network (see Figure 10) that caused it to reassess whether the released 
resources are sufficient In either instance, the master arbitration server 
automatically sends an Incremental Allocation to tiie local arbitration sender 
authorizing it additional bandv^dtii (e.g., rate) or tame on the same or a different 
channel. Conversely, tiie local arbitration server also can ask the master via an 
Incremental Request Allocation for additional bandwidth, time, channel capacity etc., 
based on the demands and performance of the local resources at its disposal. 
Again, a unique transaction ID will be exchanged as part of these request/grant 
protocols, as well as appropriate job infomiation such as file size remaining for 
transport, etc. 

Using ttiese techniques, ttie system of fliis Invention provides much greater 
flexibility for file transport. For instance, instead of requiring ttie local resources to 
be tied up while trying to complete a file transmission a set number of times, during 
which otiier files or pieces tiiereof cannot be sent over the network resource, using 
ttiis invention allows ttie local arbitration servers intelligentiy to adapt to network 
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conditions. In that manner, the local arbitration servers can decide to send a portion 
of the packetlzed file and schedule the remainder for later or decide to switch to 
another job altogether This flexibility makes greater use of local resources since the 
local arbitration servers can ensure that any "gaps' in capacity are filled with pieces 
of jobs. Meanwhile, the master arbitration server still maintains control over the 
overall use of network resources and is able to build a picture of netwoi1< usage and 
capacity based on the requests for reallocations generated by the local arbitration 
servers. 

Adaptive Bandwidth and TimeSlice Allocation (Fig. 10) 

Figure 10 details another aspect of this invention. In any communications 
network, it is important to monitor the perfonnance of various network resources, 
such as switches, routers, links and the like. This invention also adapts to changes 
in network conditions whereby the monitoring infomiation alerts the master 
arbitration server to the networi< conditions changes. By way of example, Figure 10 
schematically shows a simplified feedback loop whereby the scheduler, e.g., tiie 
master arbitration server, receives infomiation flrom a network monitoring subsystem 
1010, such as Solaris Openview or the like, ttiat provides a traffic data feed 
concerning network conditions. That information is compared witii tfie system 
optimization objectives, such as quality of service, use of capacity, etc. Together 
those parameters are taken into account by tiie master arbihnation server as it sets 
its initial schedule or bandwidtti allocation map 1020. Of course, each local 
arbitiration server also may receive a data feed describing general network 
conditions or ttie performance and capacity of ttie communications equipment 
associated with ttiat local ariDitration sen/er. 
Time Division Muitipiexing Operation 

Another mode of operation of tiie central server and Signaling System 
Servers of the present invention allows a Time Division Multiplex (TDM) mode of 
operation, giving each uplink station a periodic fixed time slice on a round robin 
basis. The duration of flie time slice may depend in each case on, among other 
things, tiie capabilities of the uplink station to begin, conduct, and end its 
transmission, but could be measured in milliseconds or microseconds. Each uplink 
station has an absolute time reference (such as GPS) which allows its exact 
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synchronization with other uplink stations. The transit time from each uplink station 
to the satellite In use Is fixed by the physical location of the uplink station relative to 
the satellite's orbit It is therefore possible for the master arbitration sen/er to tell 
each uplink station when to begin and end its transmissions such that the 
transmissions ftiom all of the uplink station in the TDM group are concatenated as 
closely as possible upon their arrival at the satellite. 

The authorizing message sent from the master arbitration server to each 
uplink station for TDM operation includes the starting time and ending time as usual, 
and also includes the exact time slice duration and period (the time from one time 
slice for this uplink station until the next). All local calculations of transmit inten^als 
are absolutely detenninistic, so that all uplink stations in the group remain In 
absolute synchronism throughout the period of the transmission without frirther 
interaction from the cental sen/er (although update messages, as mentioned above, 
are allowed). 

The advantage of this mode of operation Is ttiat it allows a TDM group of 
almost any size to share a satellite channel for real-time audio or video conferencing 
or exchange of data, with a single set of uplink / receive equipment A frequency 
division multiplexing approach (wiUi each uplink station altocated Its own bandwidth) 
requires separate equipment to receive and transmit to each participant at each 
k>cation. 

New System Protocol 

This section of ttie document will describe an embodiment of an architecture 
tiiat supports out-of-band signaling and control for transponder sharing. Using a 
single controller per channel on a transponder, uplink stations can communicate wltti 
the controller to negotiate for transponder capacity. Using TCP/IP as the signaling 
mechanism will allow for support of dialup connection as well. This should be useful 
to anyone using a motMie uplink. The controller and the uplink stations may signal 
each other using the following protocol: Note: (parenthesis indicates slate of 
parameters). 
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Table i: Basic Communicati n 


master arbitration server 


Uplink 


Idle 


Service Request (requested 
parameters) 


Service Authorized (granted 
parameters) 


(Uplink In Use) 


Heartbeat 


Heartbeat response 




Uplink finished (time expired) 


(Satelitte Resource free) 






Table 11: Transponder Busy 


master arbitration server 


Uplink 


(In use, no satellite resources 
available) 


Service Request (requested 
parameters) 


Transponder Busy (next avail...) 


Queue Request (parameters) 


(Job in Queue) 




(Transponder free) 




Service Authorized (granted 
parameters) 


(Uplink in use) 


Heartbeat 


Heartbeat response 




Uplink finished (time expired) 


(Satellite Resource free) 






Table ill: Signalinj 


a Channel Broken 


master arbitration server 


Uplink 


(Transponder in use) 


(Uplink in use) 


Heartbeat 


Heartbeat response 






Heartbeat 


(no hearO^eat no response) 


(Timeout) 


(Timeout) 


(Satellite Resource free) 


(Uplink free) 




Table IV: Addi 


tional Signals 


master arbitration server 


Uplink 


Service intenxjpted 


Uplink stopped 


(Satellite Resource free) 


(Stop transmission, Uplink free) 






Status response 


Status request 






Status request 


Status response 
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Table V: Update from Uplink 


master arbitration server 


Uplink 


(Satellite resource in use) 


Update request (new parameters) 


Changes Authorized (new parameters) 


Authorization updated 




Table VI: Update from master arbitration server 


Master arbitration server 


Uplink 


Update request (new parameters) 


Acknowledge receipt 




Notiiy Multicast group (new 
parameters) via satellite and Internet 




Receive acknowledgment from each 
member of group 




Make requested change 


Update authorization 


Acknowledge change accomplished 


Adjust satellite resource parameters 






Table VII: Other Confrol Via Web Console User Interface 


Master arbitration server 


Uplink 


Schedule Event 




Status 




Maintenance 




Logs 








IP Data 


IP Data (over signaling channel) 



Using IP communication over satellite In one direction requires an IP gateway 
at the uplink station. Communication can be to any computer system with a DVB 
satellite card and a modem for creating a return patii, altiiough skilled persons will 
recognize many ottier implementations (ttie important tiling is ttiat ttiere be some 
intelligence and a communication pipe for return messages). Eitiier point to point or 
point to multi-point (multicast) communication can be used. For multicast 
implementation, ttie service provider may utilize VNI multicast sen^ice or implement 
one of ttie methods suggested by tiie IETF to minimize reti-ansmisslon. 
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In addition to the signaling protocol, adding an API on top of TCP/IP may 
provide TCP/IP applications, mnning at the uplink station, the option selectively to 
choose which data goes over the Internet or over the high speed satellite network. 
For example the API may include an "h jwrite" (high speed write) routine which will 
use the IP over satellite if the transponder is freed. Other routines may include 
methods for query status of the satellite link. The API will include interfaces to the 
signaling protocol for requesting the usage of the transponder. The addition of this 
API will allow IP applications to request for high speed data transfer and then 
starting the data transfer possibly over the signaling channel until the capacity on the 
transponder is available. Details of the API will also be available in VNI's Shared 
Transponder Technical Specifications document. Other considerations include: 

• Signaling dropped and reconnected 

• Extend period of transponder usage which could allow for the signaling 
channel to be disconnected during extended one-way data transfer. 

• Quality of Siervice request and signaling control 

• Managing of multiple earners on a transponder. 

There are several standard controls, on the modulator, the frequency 
upconverter, and the high power amplifier, which can be used to control the output 
of the transmit link. Each makes it possible to bring the transmit link to a state of 
readiness to transmit data within milliseconds. The appropriate choice of controls is 
the one that minimizes the latency from the authorized time to begin transmission 
and the presence of the earner at the satellite. 

In addition to the potential requirement for physical switching, there will be a 
requirement for synchronization bits at the start of each transmission. A pattern of 
altemating Vs and O's similar to the start of an Ethemet data packet (10101010) will 
be required for some period of time to allow the receiver hardware to lock to a new 
transmitter on the network. As suggested in the introduction, a synchronization 
clock between transmitters will also be required. This will help eliminate too lengthy 
a signal lock delay each time a diange of control for the network is introduced. 

Out of band signaling over the Internet allows bidirectional half duplex 
communications (The Internet is used as the control link for the high speed path). 
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As a result the protocol needs to be expanded with SS7 or equivalent protocol for 
high speed communications. SS7 or the equivalent should be available, 
illustrative Applications of the Present Invention 

There are a number of potential applications of the present invention can be 
applied whenever there is a need to share a single satellite transponder or a number 
of satellite links among a network of uplinking earth stations. 
Application 1 - Scheduled transmission flirouoh a single satellite resource 

In this application a single uplinking earth station provides transmission 
sen/ices for a number of independent sources through a single satellite resource. 
The uplinking earth station is tfie only site that has access to tiie satellite resource, 
but there is contention among the sources coupled to the uplinking station for 
access time to the satellite. In this application, the control system can be used to 
prioritize and schedule access to the satellite 

In tills example digitized video data Is provided by tiie sources and is stored 
at the uplinking location in a video server The system consists of a front-end 
application allows a user to view, browse, and select files for transmission. 

Once a file has been selected for transmission tiie Multicast Sender will 
request a transmission time for ttiat file from master arbihnation server In this 
example the Multicast and flie master arbitration server may be one and ti:ie same 
since there is only one uplinking station. The Arbitrator sender prioritizes 
communications across tiie satellite data link. 

The Multicast Server uses the Internet (or anotiier suitable communications 
pathway) for communications wltti tiie receive stations or "clients." After it schedules 
a transmission wth the master ariaiti-ation server, it contacts tiie client sites through 
an open multicast group address. The open group Is contacted and ttie members of 
tiie smaller transmission group are insfructed to begin "listening" to a satellite 
resource for tine transmission. Once switched to the new closed group address each 
client in ttie newly fonned "closed group" is instiucted to register witti tiie Multicast 
Server that initiated the session. The server's IP address for this response is 
passed during the registi-ation process. This response occurs over the Internet (or 
some other communications patii for return data) and tiiat client (or receive site) can 
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now begin to receive the mutticast data being distributed in tfiat particular closed 
session. 

The output of the Multicast Server is fed to an IP gateway. Most 
manufacturers of DVB transmission cards provide an IP gateway to encapsulate the 
network traffic wittiin the MPEG2 transport data fomnat before it reaches the satellite 
transmission modulator. The modulator uses a digital modulation technique to 
modulate the data. The resulting signal is then upconverted and then amplified for 
transmission through the satellite to the clients. 

Application 2 - Access to a single transponder by multiple uplinkinq earth stations 

In this application multiple uplinking earth stations require access through a 
single satellite resource. The uplinking earth stations contend for access time to the 
satellite resource. The control system can be used manage access to the satellite 
on a non-interfering basis among a network of independent transmitters. 

In this example multiple uplinking sites are transmitting analog video 
broadcasts to a centralized location using an "occasional use" transponder. The 
control system has resented a block of time for the uplinking sites to transmit video 
back to a central location. The master arbitration server schedules access to the 
transponder by multiple sites for short periods of time wittiin this block of time. 

Each uplinking eartii station is outfitted witti a Multicast Server, which 
provides the access control at the eartii station. The Multicast Servers may access 
to the Intemet (or another communications patiiway) for communications with tiie 
master arbitration server. In addition tiie uplinking eartii station site may have 
satellite modem equipment connected to ttie Multicast Server and satellite eartii 
station transmission equipment The master arbitration server will schedule access 
for any Individual Multicast Server placed at any uplinking eartii station. The 
functionality of tiie local arbih^tion sender described above may reside on the same 
platform deploying the Multicast Server. 

The master arbitration server sends a message back to the requesting 
Multicast Server / local ariDitration server containing its transmission window. The 
master arbitration server auttiorizes the actual transmission over the satellite by 
sending an autiiorization message fertile designated satellite resource to tiie 
Multicast Server / local arbitration server tiiat is scheduled to transmit No 
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transmission may iDegin over the resource until an authorizing message is received. 
The master arbitration server uses authorizing messages as the methodology to 
insure that only one uplinking earth station is accessing ttie satellite resource at any 
given moment. 

The master arbitration server sends an authorization message, after which 
the Multicast Server / local arbitration sen/er manages the transmission of the data 
through the satellite transponder. The output of the Multicast Server / local 
arbitration server is then fed to the uplinl<ing earth stations modulation equipment 
and then it is upconverted and amplified for transmission through the satellite 
transponder. The transmissions would be received by the central location and stored 
for further processing. 

Application 3 - Access to multiple resources on a single satellite by multiple 
uplinking sites 

In tills application multiple uplinWng earOi stations require access ttirough 
multiple satellite resources on a single satellite. The uplinking earth stations contend 
for access time to the individual satellite resources. The control system can be used 
to support a variety of analog or digital transmission formats, but only one 
transmission format would be used in any specific application. For instance, the 
control system could manage multiple analog video b^nsmisslons flirough multiple 
transponders or multiple digital transmissions tiirough a single or multiple 
transponders. 

In this example, multiple uplinking sites are taBnsmitta'ng digitized video 
broadcasts to a centralized location using one of several Frequency Division 
Multiplexed (FDM) satellite diannels resident on a single satellite transponder. The 
channels are available on a fiiil time basis. The control system randomly schedules 
access to flie satellite channels by multiple sites for short: periods of time. 

Each uplinking earth station is outfitted with a Multicast Server, which 
provides the access control at ttie earUi station. The Multicast Servers would have 
access to the Internet (or another communications pathway) for communications 
witti the master ariDiti-ation server. In addition ttie uplinking eartii station sites would 
have satellite modem equipment connected to the Multicast Server and satellite 
earth station transmission equipment The master ari^ib'ation server will schedule 
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access for any individual Multicast Sender placed at any upllnking earth station. In 
this example, the Multicast Sen/er may also implement the functionality of the local 
arbitration server, particularly the ability to overlay new transfer rate or time slices 
onto a particular transmission job. 

The master arbitration server sends a message back to the requesting 
Multicast Server containing its transmission window on one of tiie available satellite 
channels. The master arbitration server authorizes flie actual transmission over the 
satellite by sending the Multicast Sen/er tiiat is scheduled to transmit an autiiorizing 
message for the designated satellite resource. The Multicast Server needs to tune 
its satellite modem to the satellite channel to which it has been assigned for the 
duration of the transmission. (An update message may cause a subsequent 
reallocation). The Uplink station must continue to transmit a carrier even if it has 
exhausted its data until its auOiorization expires. If significant time remains, it may 
send an update message to tiie master arbitration server asking for an earlier 
expiration. No Multicast Server is auttiorized to begin transmitting data over tiie 
resource unless it has received an authorization message. The confrol system uses 
autiiorization messages as tiie metiiodology to insure that only one uplinking eartti 
station is accessing tiie satellite resource at any given moment A Multicast Server 
must cease carrier transmission at the exact time specified on its current 
authorization message. 

After it has received an authorizing message firom the master arbitration 
server, ttie Multicast Server manages tiie transmission of the data tfirough the 
satellite fransponder. The output of tiie Multicast Server is then fed to ttie uplinking 
eartti stations modulation equipment that has been tuned to operate through its 
assigned satellite resource. The signal is flien upconverted and amplified for 
transmission tiirough the satellite transponder. The transmissions would be 
received by the central location and stored for furtiier processing. 
Application 4: Remote Satellite Truck Transmission 

In this application, a remote video truck contains a Signaling System Server 
and Multicast Server witii an Internet link (probably wireless, e.g. cellular). Many 
events covered are of short duration wifli an indeterminate start time, and a real time 
feed is not required. Instead of wasting satellite bandwidth waiting for the event to 
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begin, the truck can capture the event (either on tape, for an analog transponder, or 
directly digitized to the Multicast Server for a digital transponder), then request an 
authorization. When it receives the authorizing message, it transmits the segment 
At the expiration of the authorization, it powers down the earner, an "all clear" 
indication is given to the operator, and the trucl< may be moved. 
Application 5: Virtual Teleport 

Several uplink stations managed by the same master arbitration server for a 
specific satellite may have facilities to read^ satellites not available to otiier 
members of the group. For example, an East Coast uplink station may be part of the 
same group as a West Coast uplink station looking at a satellite witii continental US 
coverage. The East Coast uplink station may also have access to a European 
satellite, and the West Coast uplink station may have access to an Asian satellite. If 
master arbifa-ation servers control resources on all satellites, a virtual worldwide 
teleport may be established for scheduling resources between Europe and Asia, or 
any other two points. 

Application 6: Real Time TDM Based Telecommunication, including Video 
Conferendng 

Using the real time Time Division Multiplexing sti^tegy ouflined above, it will 
be possible to use an master arbiti^tion server to assemble groups of managed 
clients for teleconferences. Existing videoconferencing equipment may be used. 
Only a small additional delay over and above a single satellite delay (probably on the 
order of 50 milliseconds for five interactive particlpante) will be incunred. Any 
number of additional sites may participate in a receive only mode witti ttie 
appropriate Signaling System Server. 

The foregoing is provided for purposes of illustrating, explaining and 
describing several embodiments of tiie present invention. Modifications and 
adaptations to these embodiments will be apparent to tiiose of ordinary skill in the 
art and may be made wittiout departing from the scope or spirit of flie Invention and 
the following claims. Also, tiie embodiments described in this document in no way 
limit the scope of tfie below claims as persons skilled in tiiis art recognize ttiat the 
present invention can be easily modified for use with other devices and systems. 
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1 . A method for dynamically allocating network transport resources 
comprising: 

• initially allocating the bandwidth and time slice for a particular transmission; 

• providing information concerning the initial allocation to multiple, 
geographically distributed controllers, each of which are coupled to 
transmission facilities; 

• using the controllers to adjust the bandwidth or time slice in order to optimize 
file transmission over the network. 

2. A method for managing multicast file distribution over a 
communications network comprising: 

• storing at a first server computer infomiation concerning the capacity of 
channels within a networic; 

• receiving at the first server requests for access to communication channels 
within the network, wherein each request contains at least some infomiation 
relevant to setting the priority to be assigned the request; 

• forwarding to a local server infonmation assigning a selected request a 
priority, a channel, and an initial time slice and transfer rate for a particular 
request for file transmission; and 

• causing the local server to either adopt the initial time slice or transfer rate or 
to assign the request a different transfer rate, one or more different time 
slices, or both. 

3. A method according to claim 1 or 2 further comprising the step of 
contacting the recipients of the file transmission In order to confirm receipt thereof. 

4. A method according to claim 3 further comprising the step of 
contacting a group of recipients of the file to be transmitted over the networic to 
inform the recipients of a transmission time. 
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5. A method according to daim 4 in which the file transmission is 
accomplished over a satellite communications networl^ comprising at least one 
satellite and at least one uplink station. 

6. A method according to claim 4 further comprising the step of sending 
an authorizing message before any file transmission over an uplink station. 

7. The method of claim 2 further comprising the step of receiving data at 
the local server describing the capacity or perfomiance of communications 
equipment coupled to the local server. 

8. A system for allocating network transport resources comprising: 

• a master arbitration server for making at least initial resource allocations, 
which initial allocations are provided to each of 

• at least two local arfoKrataon servers, each of which communicates with a 
satellite tiransmission system and each of which is adapted to: 

a dynamically evaluate each request for transmission to determine whether 
to overiay upon tiie initial allocations an adjusted allocation changing the 
time slice, ti-ansfer rate or both associated witii each request; and 

□ communicate witii tiie master arijito^tion server regarding any changes to 
tiie time slice, bransfer rate or botii. 

9. A system of master and sub-controllers to overiay, adaptivety, 
dynamically or both, rate and time slice allocations upon a transmission job for 
transmitting a file across a first communication networtc. the system comprising: 

• at least one master artiti^ation server adapted to allocate to a selected file 
transmission job an initial channel, time slice and rate; and 

• two or more local arbiti-ation servers adapted to communicate wth the master 
arbiti-ation server and, based upon tiie priorities of the jobs assigned to each 
local arbitration server, assign different rate, time slice or channel information 
to a selected job in order to ensure effident use of resources In tiie first 
communication networic. 
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10. A system according to daim 9 wherein the communication between 
the local and master arbitration servers is conducted over a second communications 
network. 
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